Repository navigation
fix(auth): honor connector HTTP settings in OAuth M2M - #562
vuanhphung wants to merge 14 commits into
Conversation
Bind a connector-local authenticator after options are finalized, sharing the base transport and User-Agent with discovery and token grants. Preserve token refresh configuration without mutating reused authenticators. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
Share token and endpoint state across connector-bound authenticators while fetching through the calling connector HTTP client. Cover shared and preused tokens, concurrent refresh, TLS verification, nil hooks and OAuth errors; clarify upgrade behavior. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
Document the shared-authenticator transport wait trade-off identified in the converged adversarial review. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
64ba30e to
2877915
Compare
Mark discovery and grant contexts with their shared authenticator cache so transports can reapply authentication without relocking or replacing grant credentials. Cover reentry, marker scope and refresh; clarify queued failed grant delays and credential-exchange transport/TLS settings. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
Limit built-in OAuth hostname relaxation to the connector host while preserving chain verification, SQL behavior and caller-owned transports. Validate discovery endpoints before shared caching and cover TLS policy, account-rooted discovery and authenticator-copy cache isolation. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
Compose the existing OAuth token-request cap with connector-local discovery, grants and refreshes. Preserve shared caches, reentrancy markers, TLS verification and endpoint validation; retain the timeout regressions and add combined deadline coverage. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Low
Looks good, with one low concern. I traced the cache/marker re-entrancy, the cloned OAuth TLS config with chain-only verification, and the discovery client copy, and they look correct. Endpoint validation newly rejects some hosts that cloud inference still accepts.
Partition token and discovery caches by transport trust while retaining the authenticator-family reentrancy guard and request timeout cap. Keep built-in Thrift OAuth fully verified, restore configured Azure host compatibility, and document backend-specific TLS behavior and custom-transport credential exposure. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Low
Mostly solid: 1 medium concern (unbounded per-transport token-cache map). The PR description says WithSkipTLSHostVerify relaxes the OAuth check for the connector hostname. The code, CHANGELOG and tests all keep full OAuth verification, so the description should be updated to match.
Reapply the bound authenticator on SQL requests and the original authenticator on credential requests, exercising cache sharing and the family reentrancy marker without crossing trust profiles. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Low
Mostly solid, with 1 medium concern: the shared cache map keyed by transport keeps every new transport alive. Discovery validation, the TLS hardening and the recursive-auth marker look correct, and tests cover them well.
Share only the default verified built-in cache; keep caller-transport and custom-TLS caches connector-local without retaining transport keys. Cover repeated connector creation, preserve isolation and reentry, and disclose that global HTTP overrides no longer apply to connector OAuth. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
|
The PR description now records the maintainer's post-round-1 R1-3 decision: built-in Thrift OAuth always verifies chain and hostname, superseding both original Finding 1 options. The stale name-skip wording is gone. Adversarial round 2 converged; the remaining cache-retention/disclosure items are fixed in 0d3b7f9, and redirects remain the accepted follow-up. This comment was generated with GitHub MCP. |
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Low
Looks good, with 1 low concern. The cache partitioning, recursion marker, timeout composition and TLS hardening all check out against PooledTransport and the m2m code paths. The custom-TLS cache partition can't be reached through the public config today, since TLSConfig only ever carries MinVersion or the skip flag.
Normalize a single DNS root dot only when validating discovered endpoint hosts, preserving metadata issuer fallback checks and original advertised URLs. Cover case, ports and invalid host suffixes, including complete OAuth grants over local TLS. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Low
One medium concern. The transport, User-Agent, timeout and cache-partition wiring looks correct and is well tested. The new discovery-endpoint validation breaks AWS/GCP alias hosts, while the Azure equivalent still works.
Clarify that the approved Databricks credential-host suffix policy rejects non-Databricks endpoints advertised by AWS/GCP aliases, including same-host endpoints, while retaining the configured Azure-host exception. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
Preserve connector attribution and request bounds without exposing OAuth to caller SQL transports. Share one token/config cache and lock across authenticator copies to restore startup single-flight. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
|
Updated in dbb9860 to address both maintainer findings. OAuth discovery, grants and refreshes now always use the driver's verified, environment-proxy/system-root transport, never the caller's SQL transport: host rewrites/private TLS settings must not break cross-host issuers or expose the client secret to caller hooks. All copies of one authenticator again share one cache and lock, avoiding N discoveries/grants and restoring concurrent-startup single-flight. Connector User-Agent, shorter client timeout and existing request caps remain; SQL/CloudFetch routing and kernel behavior are unchanged. Updated docs, regression tests, live checks and benchmark results are in the PR body; the follow-up now awaits adversarial review. This comment was generated with GitHub MCP. |
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 High · 1 Low
Looks good, with one low concern. The code, README and CHANGELOG agree: OAuth uses a verified driver transport and authenticator copies share one cache. Two claims in the PR description don't match the code: that WithTransport carries OAuth, and that each caller-transport connector owns its cache. Please update the description.
This overview is Claude Opus 5.5's initial take; some of its points were set aside by the cross-check judge and GPT-6.1 Sol added findings it doesn't mention — see the findings below.
Other findings
- 🟠 High — Disputed by the cross-check judge (it quotes
cache: c.cache,); a human should decide. Original finding: Every HTTP-client binding shares token/config state with the standalone authenticator, so a token or discovery configuration obtained through an unverified custom/default client can bypass discovery and TLS verification in a subsequently bound strict connector. Separate standalone and custom-client caches from the verified built-in partition, and add a custom-first/strict-second regression instead of asserting unconditional cache sharing. (Found by GPT-6.1 Sol · disputed by the judge)
🔀 1 finding considered and dismissed by the judge
- 🟡 Medium
internal/client/client.go:657— Whenever WithTransport is set, OAuth replaces it with a fresh default transport, so connector-specific proxies, custom roots, mTLS, and tracing still do not rea
Judge's evidence:if cfg.Transport != nil || (cfg.TLSConfig != nil && cfg.TLSConfig.InsecureSkipVerify) { oauthBase = PooledTransport(&con…
Consolidate routing, timeout, cache and TLS checks around one fixture. Remove dropped-design guards, subprocess tests and the non-gating benchmark while retaining the maintainer-required endpoint policy. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
✅ No issues identified.
🔀 1 finding considered and dismissed by the judge
- 🟡 Medium
auth/oauth/oauth.go:86—validateEndpointonly accepts endpoints whose host ends in a Databricks suffix. A token endpoint on the user's own configuredhostNamegets rejected too, so
Judge's evidence:// Suffix matching (not the substring matching InferCloudFromHost uses for cloud
Apply D29 and round-one review: restore main's discovered endpoints, share one verified OAuth pool independently of SQL, and test actual driver transport routing. Simplify attribution coverage and document the context-client contract. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
M2M discovery, grants and refreshes now send the connector User-Agent through one process-wide, fully verified driver HTTP pool, independent of SQL. It uses environment proxies and system roots (
SSL_CERT_FILE);http.DefaultClient/http.DefaultTransportoverrides no longer apply. Existing request/discovery caps and per-authenticator cache sharing remain; no public timeout option is added.Verification:
make test1708 tests, 9 credential-gated skips; lint 0 issues; auth/client/root race, focused race ×10, Go1.27.3 kernel build/focused tests pass. Live Thrift 4/4, kernel 3/3, unified-host M2M 2/2 variants, no skips; successful SP credentials remain unavailable and native UA is not wire-observed. Routing/deadlock mutations fail their own assertions without cleanup panics. Generated formatting exceptions match main. Historical no-network grant benchmark (unchanged grant hot path): 6.017 → 6.548µs, 71 → 82 allocations, 18 samples/revision; no new timing claim.Per-file scope against main c1f2469: +524/-42 → +400/-37
CHANGELOG.mdREADME.mdauth/oauth/m2m/m2m.goauth/oauth/m2m/m2m_test.goauth/oauth/oauth.goauth/oauth/oauth_test.goconnector.goconnector_test.gointernal/client/client.gointernal/client/oauth_m2m_test.gointernal/client/oauth_test.goProduction scope:
m2m.gobinds connector HTTP clients while sharing token/config/lock (#413/D28);oauth.gothreads that client through bounded discovery (#413/#557).client.goadds the shared OAuth pool and cloned-request UA decorator (#413/R1-1). Exported helper/option docs describe these contracts. Tests exercise the real driver transport, single-flight and actual bound-client attribution.Cross-driver state
AllowSelfSignedCerts=0(ClientConfigurator.java:58,129,418@840f3a4).auth/oauth.py:340,unified_http_client.py:105@087b2bf).internal/client/client.go:645,m2m.go:49@3609c63).WithTransport(kernel_config.go:156,202@c1f2469).rejectUnauthorized=true(HttpConnection.ts:164,OAuthManager.ts:62@7667e36).http.rs:258,280,534,oidc.rs:114@8877579).Adversarial review: round 1 converged (open-high 0); R1-1/2/4/5 fixed in 3609c63, R1-3 moot per D29. Bot approved 3609c63 with no findings; CI is green. Redirect policy and serialized-failure queue delays remain pre-existing follow-ups.
Fixes #413
This pull request and its description were written by Isaac.